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Description 

SCALING DEVICE AND METHOD FOR 
SCALING A DIGITAL PICTURE 

Background of Invention 
[000 1 ] 1. Field of the Invention 

[0002] The present invention relates to a scaling device and 

method for scaling a digital picture, and more particularly, 
to a scaling device and method for scaling a digital picture 
where the horizontal-scaling and vertical-scaling pro- 
cesses to be performed in one-pass and only uses a small 
amount of buffer memory. 

[0003] 2. Description of the Prior Art 

[0004] The conventional method of digital picture scaling is to 
separate a 2 dimensional (2D) scaling process into two 
ID-scaling processes. In other words, the scaling process 
is first to be performed on the horizontal direction (width) 
and then on the vertical direction (height). 

[0005] Theoretically, the method to perform ID-scaling process 



[0006] 



[0007] 



[0008] 



is to exploit the sampling formula for a new sampling 
point by the following equation: 



where x(t) is the pixel value at a new sampling point t 
from n=0 and the range oft is 0<t<l, x(n) is the original 
pixel value at index n, and h(t-n) is the value of an inter- 
polation function inversed and shifted by t from index n. 
Furthermore, the coefficients i and j give the number of 
original pixels involved in interpolating x(t), i.e. the num- 
ber of original pixels involved is given by (i+j) which gives 
the number of filter taps needed and h(n) is the tap 
weighting at index n. 

The conventional scaling method has two drawbacks. 
First, the two-pass process of the conventional scaling 
method is not suitable for real-time applications, because 
the vertical scaling process cannot be performed until the 
horizontal scaling process has been accomplished, or vice 
versa. Furthermore, due to the two-pass process, the 
conventional scaling method requires a buffer memory to 




store the results from the horizontal scaling, as well as to 
provide the freedom to use interpolation filters of any 
length to achieve the required scaling quality. 
[0009] FIG. 1 shows the conceptual diagram for the up-scaling 

process, where W and H are the old width and the old 

K old old 

height of the original digital image, and W and H are 

new new 

the new width and new height of the image after scaling. 
The buffer memory is required to store the results from 
scaling the pictures horizontal dimension and the size of 
the buffer is (W x H ) bytes. 

old old 1 

[0010] There are multiple drawbacks in the conventional scaling 
method especially when up-scaling an input picture. As- 
sume the input source image is to be up-scaled to two 
times larger in each direction, the buffer memory required 
becomes (W x H ) = 2 x (W x H ). Furthermore 

new new old old 

the delay time of the conventional scaling method is not 
feasible for some applications. In particularly, the high 
memory requirement is not suitable for the hardware im- 
plementation by integrated circuits (ICs) and the high data 
delay time is not suitable for real time applications. 
[0011] jo solve the aforementioned problems, the common 

practice is to trade the vertical scaling quality with low- 
buffer-memory solutions, which in turns reduces the data 



delay time. For example, the vertical scaling process uses 

only a two-tap filter so that only a two-line buffer needs 

to be maintained. However, the small buffer limits the 

scaling quality and a device and method for high quality 

scaling in real time using little memory is needed. 
Summary of Invention 

[0012] it j S therefore an advantage of the claimed invention to 
provide a scaling device and method to solve the afore- 
mentioned problems. 

[0013] According to the claimed invention, the scaling device for 
scaling a digital picture comprises a source buffer for 
storing the digital picture, a processing unit for creating 
two weighting matrices, an intermediate buffer for storing 
output data generated by multiplying a block of the digital 
picture by one of the weighting matrices, and a destina- 
tion buffer for storing output data generated by multiply- 
ing the output data stored in the intermediate buffer by 
the other weighting matrix. 

[0014] Further, the method for scaling a digital picture which has 
a plurality of blocks comprises inputting the digital pic- 
ture, creating two weighting matrices, and multiplying the 
plurality of blocks of the digital picture by the weighting 
matrices. 



[0015] These and other objectives of the claimed invention will 

no doubt become obvious to those of ordinary skill in the 

art after reading the following detailed description of the 

preferred embodiment that is illustrated in the various 

figures and drawings. 
Brief Description of Drawings 

[0016] FIG.l is a diagram of a prior art scaling method. 

[0017] FIG. 2 is a conceptual diagram of a scaling method ac- 
cording to the present invention. 

[0018] FIG. 3 is a flowchart for creating a weighting matrix ac- 
cording to the present invention. 

[0019] FIG. 4 is a flowchart of the scaling method in Fig. 2. 

[0020] FIG. 5 is the segmented flowchart for initialization of the 

scaling method according to one embodiment of the 

present invention. 
[0021] FIG. 6A is the segmented flowchart for vertical scanning of 

the scaling method according to one embodiment of the 

present invention. 
[0022] FIGS. 6B and 6C show boundary conditions of different 

options according to the present invention. 
[0023] FIG. 7 is the segmented flowchart for horizontal scanning 

of the scaling method according to one embodiment of 



the present invention. 
[0024] FIG. 8 is the segmented flowchart for performing the scal- 
ing method according to one embodiment of the present 
invention. 

[0025] FIG. 9 is a block diagram of a scaling device for scaling a 
digital picture according to one embodiment of the 
present invention. 

[0026] Fig. 10 is a schematic diagram of the processing unit in 

FIG. 9 for performing the matrix multiplication of the 

present invention. 
Detailed Description 

[0027] FIG. 2 shows a conceptual diagram of a scaling method 
200 of the present invention applied for the up-scaling 
process. In the method 200, a digital picture 210 with a 
dimension of Hold-by-Wold is first received. The digital 
picture 210 is filtered by an interpolation filter with a 
length of m used for vertical scaling and by an interpola- 
tion filter with a length of n used for horizontal scaling to 
sequentially generate a plurality of blocks 220 each hav- 
ing a size of m-by-n. Each of the blocks 220 is multiplied 
by two weighting matrices to form a scaled picture 230 
with a dimension of H -by-W . The same concept can 

new new 

also be adapted for the case of the down-scaling process. 



[0028] The present invention reduces the complexity of imple- 
menting Eq.l. This invention exploits two weighting ma- 
trices to store the weights of filter taps. These weights are 
pre-calculated at the precision that is adjustable to match 
the requirement of different systems. In other words, the 
range of t in Eq.l is divided into a number of segments 
and each segment represents all the sampling points that 
fall into this segment range. Furthermore, this invention 
provides the freedom to use a variable number of filter 
taps that also depends on the system requirement. As a 
result, if the system implementing this invention requires 
high precision, the range of t can be finely divided into a 
large number of segments and more number of filter taps 
can be used. On the other hand, if the system has a con- 
strained resource, then the range oft can be divided into 
a smaller number of segments and fewer number of filter 
taps can be used. 

[0029] FIG. 3 shows a flowchart of creating a weighting matrix. 
Assuming the range oft is divided into H segments, the 
number of filter taps is L, the scaling factor is s, and two 
weight factors are a and b. In Step 320, s, a, b, H and L 
are inputted. In step 330, if the scaling adjusting factor 
flag AdjustEn=0 (disable), then in step 350 if s<l, the ad- 



justed scaling factor s =1, otherwise s =s. Whereas, if 

a a 

AdjustEn=l (enable), then in step 340 if s<l, the adjusted 
scaling factor s =a, otherwise s =s*b. In step 360, the 

a a 

weighting matrix WeightMat is generated as 
[0030] WeightMat(i,j)= 



[0031] where i ranges from 0 to (H-l), j ranges from 0 to (L-l), 
w(i,j) is the pre-normalized weighting at (i,j), and W(i,j) is 
the normalization factor given by 

[0032] 





l-l 




J-0 



[0033] The pre-normalized weighting w(i,j) is given by 

[0034] 



J 



P-J + 



H 



[0035] where h(s) is an interpolation filter, p=L/2-l, H is the 
number of segments of t, and s is the adjusted scaling 

a 

factor. 

[0036] FIG. 4 shows a flowchart of the scaling method of the 

present invention. In step 410, the digital picture 210 is 
inputted and stored in a source buffer. The size of the 
source buffer is at least the size of the width of the digital 
picture 210 times the height of the digital picture 210. In 
step 420, the horizontal and vertical scaling factors are 
determined from the final required image dimension of 
the user. The user enters the required image dimension 
and the scaling factors are automatically determined by 
dividing the width and height of the digital picture 2 10 by 
the width and height of the scaled picture 230 respec- 
tively. With the scaling factors, two weighting matrices are 



generated according to the flowchart in FIG. 3. In step 
430-450, the new sampling points are scanned according 
to the scaling factors. The scanning of the new sampling 
points at those new sampling points includes the follow- 
ing steps: in step 430, determining the indices of pixels 
used for interpolation and adjusting those indices accord- 
ing to the boundary conditions; in step 440, transferring a 
block 220 of the digital picture 210 to the block buffer 
according to the indices; in step 450, performing horizon- 
tal and vertical scaling by matrix multiplications; and out- 
putting results to a destination buffer in step 460. Finally 
in step 470, these steps 430-460 are repeated until all 
the new sampling points have been generated. If all the 
new sampling points are generated, the flow ends. If not 
all the new sampling points are generated, the flow re- 
verts to step 430 and repeats steps 430-450. 
[0037] Th e preferred embodiment of the present invention 
chooses 16 segments (i.e. H = 16) and 4 filter taps (i.e. 
L=4) for demonstrating the best mode of the present in- 
vention. However the number of segments and filter taps 
is merely a design choice and presents no limitation in 
anyway to the present invention. The detailed flowchart 
of the present invention is divided into 3 separate parts 



which are the initialization in FIG. 5, the vertical scaling in 
FIG. 6A, the horizontal scaling in FIG. 7, and finally, Fig. 8. 
[0038] please refer to FIG. 5, the initialization process begins 
with first inputting the digital picture 210 into a source 
buffer SRC in step 5 10. The size of the digital picture 2 10 
is denoted by W and H and the size of the source 

1 old old 

buffer SRC should be sufficient to accommodate the digi- 
tal picture 210. In step 520, the user enters the required 
new image size W and H and the horizontal and ver- 

new new 

tical scaling factors are respectively determined by the 
following equations. 



[0039] 




and 




[0040] where W and H are respectively the width and height 

old old 



of the digital picture 210 and W and H are respec- 

new new 

tively the width and height of the scaled picture 230. Once 
the scaling factors are determined, the horizontal and 
vertical weighting matrices WeightMatl and WeightMat2 
are generated according to the flowchart in FIG. 3 with the 
formulas given earlier. In the present embodiment, the 
size of both the WeightMatl and WeightMat2 is 16x4 be- 
cause H = 16 and L=4. It is to be noted that the size of the 
weighting matrices is dependent on the required resolu- 
tion which can be adjusted by changing the values of H 
and L. A larger H value means the range of t is divided 
into more segments which means that the resolution of 
the scaling is higher. A larger L value corresponds to a 
greater number of filter taps, which means more sur- 
rounding pixels are referenced. In creating the Weight- 
Matl and WeightMat2, the scaling factor adjusting flag 
AdjustEn is either set to 0 (disable) or 1 (enable) depend- 
ing on the application. At the end of the initiation process 
in step 530, a new vertical index y2 is first set to 0 before 
the vertical scaling begins. The flowchart in FIG. 5 con- 
nects to a junction A which reappears in FIG. 6A used only 
to illustrate the continuity of the entire process. 
[0041] please refer to FIG. 6A, which shows the vertical scanning 



process. The flowchart is a continuation from FIG. 5 
through junction A. In step 610, the relationship between 
an old vertical index yl of the input source image and the 
new vertical index y2 of the scaled image is defined by the 
following equation: 

[0042] y l = s * y2 

V 

[0043] y =fioor(yl) 

[0044] wi_v=round[(yl-y)xH] 

[0045] where s^ is the vertical scaling factor, y2 is the new verti- 
cal index of the scaled image, yl is the old vertical index 
of the input source image that corresponds to y2,y is the 
integer part of yl,wi_v is the index to extract filter tap 
weightings from WeightMatl for the vertical scaling pro- 
cess. The function floor() and round () are a flooring pro- 
cess and rounding process respectively. The flooring pro- 
cess takes the old vertical index and sets it as the current 
vertical index y for calculation purposes. The rounding 
process takes the decimal answer from the calculation and 
rounds off to the closest integer value. In step 620, a 
safely check is performed to determine if wi_v equals to H 
because when such happens the pointer wi_v exceeds the 
valid range of 0 to H-l so no weighting values are able to 



be fetched. If wi_v=H, the flowchart sidetracks to step 630 
to reset the value of wi_v to 0 and increment the value of 
the current vertical index by 1 to continue with the vertical 
scaling process. If wi_v is not equal H, the flowchart pro- 
ceeds to step 640 where the vertical index pointers are 
determined from the inputted source image and according 
to the number of filter taps. In this embodiment, the 
number of filter taps is 4 so there are 4 vertical index 
pointers which are iO, il, \2, and i3. The 4 vertical index 
pointers are respectively set based on the boundary con- 
ditions which are generalized by the following general 
formulas:i[p + abs(pO)] = ((x + p)*e + S*a + ((W-l) - d + 
S)*b) * ((-l)Aa),where i[n] is the data retrieval for tap 
number n and n ranges from 0 to L-l, and W is the data 
length. The other parameters are given as 
[0046] n = p + abs(pO), 

[0047] p o = -(L/2-1), 

[0048] p i = 1/2, 

[0049] a = sign(x+p), 

[0050] b = sign((W-l)-(x+p)), 

[0051] C = W-l-X, 



[0052] d = X + p-(W-l), 

[0053] e = N ot(b) & 1, 

[0054] s= o or 1. 

[0055] 



sign{X) = 



1, X<0; 
0, X>=0. 



[0056] Function abs(X) gives the absolute value of X. 

[0057] Function Not(X) is a bit-wise operation and inverts the bi- 
nary bits, ie. 
[0058] N ot(0) = 1 and Not(l) =0. 

[0059] one should note that although parameters a and b used 
here are the same as those in Fig. 3, they represent differ- 
ent meaning. 

[0060] The vertical index pointers are created from the current 

vertical index according to the boundary conditions of op- 
tion 1. In this embodiment, 

[0061] if y =o, then i0=y, il=y, i2=y+l, and i3=y+2 

[0062] |f y =H 2, then i0=y-l, il=y, i2=y+l, and i3=y+l 

old- 

[0063] |f y =H -1, then i0=y-l, il=y, i2=y, and i3=y-l 

old 



[0064] otherwise, i0=y-l, il=y, i2=y+l, and i3=y+2 

[0065] After all the vertical index points are determined, step 640 
ends and step 650 follows to set x2 to 0. The flowchart in 
FIG. 6A connects to a junction B which reappears in FIG. 7 
used only to illustrate the continuity of the entire process. 
Junction F in FIG. 6A represents an action reverted back 
from FIG. 7 as part of the loop. Vertical scanning of the 
source image to generate vertical index pointers is com- 
pleted. 

[0066] please refer to FIG. 7 which shows the horizontal scanning 
process. The flowchart is a continuation from FIG. 6A 
through junction B. The horizontal scanning process is 
identical to the vertical scanning process so the detailed 
description can be referred to the vertical scanning pro- 
cess. 

[0067] X l = s * x2 
h 

[0068] x =floor(xl) 

[0069] wi_h = round[(xl-x)xH] 

[0070] where s h is the horizontal scaling factor, x2 is the new 
horizontal index of the scaled image, xl is the old hori- 
zontal index of the input source image that corresponds 
to x2, x is the integer part of xl, wi_h is the index to ex- 



tract filter tap weightings from WeightMat2 for the hori- 
zontal scaling process. 

[0071] The horizontal index pointers are created from the current 
horizontal index according to the boundary conditions. In 
this embodiment, 

[OO 72 ] If x=0, then i0=x, il=x, i2=x+l, and i3=x+2 

[0073] |f X =W -2, then i0=x-l, il=x, i2=x+l, and i3=x+l 

old 

[0074] |f x =W -1, then i0=x-l, il=x, i2=x, and i3=x-l 

old 

[0075] otherwise, i0=x-l, il=x, i2=x+l, and i3=x+2 

[0076] After all the horizontal index points are determined, step 
730 is finished and the horizontal scanning is completed. 
The flowchart in FIG. 7 connects to a junction C which 
reappears in FIG. 8 used only to illustrate the continuity of 
the entire process. Horizontal scanning of the source im- 
age to generate horizontal index pointers is completed. 

[0077] please refer to FIG. 8, which shows the engine for per- 
forming the scaling method of the present invention. In 
step 810, data of the segments that are defined by the 
first block according to the vertical index pointers and the 
horizontal index pointers is transferred to the block buffer 
B for processing. In this embodiment, the number of filter 
taps is chosen to be 4 for both the vertical filter taps and 



the horizontal filter taps. Therefore the buffer size is 4x4 
which equals 16 pixel-unit. If each pixel is 1 byte, the size 
of the block buffer is 16 bytes. The block buffer B can be 
implemented in many different ways according to the sys- 
tem requirement but common buffer memory comprises 
DRAM, SDRAM, flash memory, and the like in DSCs and 
comprises registers made by flip-flops, register-file (RF), 
and the like in ICs. Furthermore the 2D block buffer can 
be implemented by a ID line buffer by using proper in- 
dexing method to mimic a 2D block buffer. The block of 
data is transferred to the block buffer B according to the 
vertical index pointers and the horizontal index pointers. 
The matrix of data in the block buffer is as follow: 



[0078] 


B(0,0)=SRC(j0,i0) B(1,0)= 


=SRC(jl,i0) B(2,0)= 


=SRCG2,iO) 


[0079] 


B(3,0)=SRC(j3,i0) 






[0080] 


B(0,l)=SRC(j0,il) B(l,l)= 


=SRC(jlJD B(2,l)= 


=SRC(j2,il) 


[0081] 


B(3,l)=SRC(j3,il) 






[0082] 


B(0,2)=SRC(jO,i2) B(l,2)= 


=SRC(jlJ2) B(2,2)= 


=SRC(j2,i2) 


[0083] 


B(3,2)=SRC(j3,i2) 






[0084] 


B(0,3)=SRC(j0,i3) B(l,3)= 


=SRCG1J3) B(2,3)= 


=SRC(j2,i3) 


[0085] 


B(3,3)=SRC(j3,i3) 







[0086] | n step 830, the filter weightings are transferred respec- 
tively from WeightMatl and WeightMat2 to create two 
vectors W and W using index wi_v and wi_h. 

v h a 

[0087] The values of are created according to the following: 
[0088] w (0)=WeightMatl(wi_v, 0) 

V 

[0089] w (l)=WeightMatl(wi_v, 1) 

V 

[0090] w (2)=WeightMatl(wi_v, 2) 

V 

[0091] w (3)=WeightMatl(wi_v, 3) 

V 

[0092] and the values of W are created according to the follow- 

h 3 

ing: 

[0093] w (0)=WeightMat2(wi_v, 0) 

h 

[0094] w (i)=WeightMat2(wi_v, 1) 

h 

[0095] w (2)=WeightMat2(wi_v, 2) 

h 

[0096] w (3)=WeightMat2(wi_v, 3) 

h 

[0097] After the corresponding data is entered in the block buffer 
B and the vectors W and W are created, in step 850, a 

v h 

matrix multiplication of the data in the block buffer B and 
both the vectors W and W is performed. The W is dot 

v h v 

multiplied by the matrix of data in the block buffer B and 
dot multiplied by the W h _ The output of the matrix multi- 



plication is a 2D result which is stored in the destination 
buffer DST. The mathematical formula below shows that 
how the values in the destination buffer DST is calculated 
from the block buffer. 
[0098] DST(x2,y2)= W BW 



w,(2) wM 



'5(0,0) 5(1,0) 5(2,0) 5(3,0)' 

5(0,1) 5(1,1) 5(2,1) 5(3,1) 

5(0,2) 5(1,2) 5(2,2) 5(3,2) 

5(0,3) 5(1,3) 5(2,3) 5(3,3) 



WO) 
WiCl) 

WiC5 



[0099] where DST(x2,y2) is the output picture memory buffer and 
DST(x2,y2) is the interpolated pixel value at coordinates 
(x2,y2). A checking process is performed to determine if 
the scaling process is completed by determining if the 
new horizontal index x2 is smaller than the W in step 

new 

870 and if the new vertical index y2 is smaller than the H 

new 

in step 890. If the value of x2 is smaller than the W , the 

new 

flow proceeds to step 860 where x2=x2 + l and reverts 
back to step 710. Similarly, if the value of y2 is smaller 
than the H , the flow proceeds to step 880 where 

new 

y2=y2 + l and reverts back to step 610 in FIG. 6A. 
[0100] please refer to FIG. 9, which shows a block diagram of 

scaling device 1000 for scaling a digital picture according 
to the present invention. The scaling device 1000 com- 
prises a source buffer 1010 for storing a digital picture, a 



processing unit 1015 for creating two weighting matrices, 
and an image divisor 1020 for filtering the digital picture 
to generate a plurality of blocks of the digital picture, a 
block buffer 1030 for sequentially storing the plurality of 
blocks of the digital picture generated by the image divi- 
sor 1020, an weighting matrix buffer 1035 for storing the 
weighting matrices, and a destination buffer 1040 for 
storing the scaled digital picture. 
[0101] please refer to FIG. 10, which shows a schematic diagram 
of the processing unit 1015 in FIG. 9 for performing the 
matrix multiplication of the present invention. The pro- 
cessing unit 1015 comprises a plurality of multipliers and 
adders electrically coupled to the block buffer 1030. Each 
data unit in the block buffer 1030 is electrically coupled to 
a multiplier which is further electrically coupled to receive 
W . The data units in the first column of the data in the 

h 

block buffer B where i=0 are electrically multiplied by W 
h (0), the data units in the second column of the data in the 
block buffer B where i = l are electrically multiplied by W 
(1), the data units in the third column of the data in the 

h 

block buffer B where i=2 are electrically multiplied by W 
h (2), and the data units in the fourth column of the data in 
the block buffer B where i=3 are electrically multiplied by 



W (3). The outputs from the multipliers are electrically 

h 

coupled to an adder before being sent to another multi- 
plier to perform the multiplication with W . The output 

V 

from the multipliers are taken row by row, where the data 
units in the first row of the data in the block buffer B 
where j=0 are electrically are multiplied by W v (0), the data 
units in the first row of the data in the block buffer B 
where j = l are electrically are multiplied by W (1) the data 
units in the first row of the data in the block buffer B 
where j=2 are electrically are multiplied by W v (2), and the 
data units in the first row of the data in the block buffer B 
where j = 3 are electrically are multiplied by W v (3). 

[0102] Compared with the prior art, the buffer memory can be 
greatly reduced even when up-scaling a digital picture. 
Further, the vertical and horizontal scaling processes are 
performed for each block before scaling another block 
thus allowing for real-time applications. Moreover, one 
should note that the present invention can be applied to 
pictures with a single color component and pictures with 
multiple color components. 

[0103] Those skilled in the art will readily observe that numerous 
modifications and alterations of the device may be made 
while retaining the teachings of the invention. Accord- 



ingly, that above disclosure should be construed as lim- 
ited only by the metes and bounds of the appended 
claims. 



